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The purpose of this digest is to offer tips in designing quality questionnaires and on 
avoiding common errors. Some of the more prevalent problems in questionnaire 
development are identified and suggestions of ways to avoid them are offered. 

Do keep the questionnaire brief and concise. Some questionnaires give the impression 
that their authors tried to think of every conceivable question that might be asked with 
respect to the general topic of concern. The result is a very long questionnaire causing 
annoyance and frustration on the part of the respondents resulting in non-return of 
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mailed questionnaires and incomplete or inaccurate responses on questionnaires 
administered directly. To avoid this first potential problem the investigator must define 
precisely the information desired and endeavor to write as few questions as possible to 
obtain it. Peripheral questions and ones to find out "something that might just be nice to 
know" must be avoided. A clear-cut need for every question should be established. 

Do get feedback on your initial list of questions. Feedback may be obtained from a small 
but representative sample of potential responders. A field trial of a tentative form of the 
questionnaire is also desirable. 

Do locate personal or confidential questions at the end of the questionnaire. The early 
appearance of unsettling questions may result in respondents discontinuing the 
questionnaire. 

Do order categories. When response categories represent a progression between a 
lower level of response and a higher one, it is usually better to list them from the lower 
level to the higher in left-to-right order, for example, 

1) Never 2) Seldom 3) Occasionally 4) Frequently 

Do consider combining categories. In contrast to the options listed just above, consider 
the following: 

1) Seldom or never 2) Occasionally 3) Frequently 

Combining "seldom" with "never" might be desirable if responders would be very 
unlikely to mark "never" and if "seldom" would connote an almost equivalent level of 
activity, for example, in response to the question, "How often do you tell you wife that 
you love her?" In contrast, suppose the question were, "How often do you drink 
alcoholic beverages?" Then the investigator might indeed wish to distinguish those who 
never drink. When a variety of questions use the same response scale, it is usually 
undesirable to combine categories. 

Do ask responders to rate both positive and negative stimuli. There is sometimes a 
difficulty when responders are asked to rate items for which the general level of 
approval is high (the "apple pie" problem). There is a tendency for responders to mark 
every item at the same end of the scale. By offering positive and negative responses 
the respondent is required to evaluate each response rather than uniformly agreeing or 
disagreeing to all of the responses. 

Do choose appropriate response category language and logic. The extent to which 
responders agree with a statement can be assessed adequately in many cases by the 
options: 

1) Agree 2) Disagree 
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However, when many responders have opinions that are not very strong or well-formed, 
the following options may serve better: 

1) Agree 2) Tend to agree 3) Tend to disagree 

4) Disagree 

These options have the advantage of allowing the expression of some uncertainty. In 
contrast, the following options would be undesirable in most cases: 

1) Strongly agree 2) Agree 3) Disagree 
4) Strongly Disagree 

Some would say that "Strongly agree" is redundant or at best a colloquialism. In 
addition, there is no comfortable resting place for those with some uncertainty. 

Avoid open-ended questions. In most cases open-ended questions should be avoided 
due to variation in willingness and ability to respond in writing. 

Avoid the response option "other." Careless responders will overlook the option they 
should have designated and conveniently mark the option "other" or will be hairsplitters 
and will reject an option for some trivial reason. An exception to the foregoing advice is 
any case in which the categories are clear-cut, few in number, and such that some 
responders might feel uncomfortable in the absence of an applicable response. 

Avoid category proliferation. A typical question is the following: 

Marital status: 1) Single (never married) 2) Married 

3) Widowed 4) Divorced 5) Separated 

Unless the research in question were deeply concerned with conjugal relationships, the 
distinctions among all of these categories are not useful. Usually, such a question 
reflects the need to distinguish between a conventional familial setting and anything 
else. If so, the question could be: 

Marital status: 1) Married and living with spouse 

2) Other 

Avoid scale point proliferation. In contrast to category proliferation, which seems usually 
to arise somewhat naturally, scale point proliferation takes some thought and effort. An 
example is: 

1) Never 2) Rarely 3) Occasionally 4) Fairly often 
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5) Often 6) Very often 7) Almost always 8) Always 

Such stimuli run the risk of annoying or confusing the responder with hairsplitting 
differences between the response levels. Psychometric research has shown that most 
subjects cannot reliably distinguish more than six or seven levels of response. Offering 
four to five scale points is usually quite sufficient to stimulate a reasonably reliable 
indication of response direction. 

Avoid responses at the scale mid-point and neutral responses. The use of neutral 
response positions had a basis in the past when crude computational methods were 
unable to cope with missing data. In such cases, non-responses were actually replaced 
with neutral response values to avoid this problem. The need for such a makeshift 
solution has long been supplanted by improved computational methods. Consider the 
following questionnaire item: 

The instructor grades fairly. 

1) Agree 4) Tend to disagree 

2) Tend to agree 5) Disagree 

3) Undecided 

There is no assurance whatsoever that a subject choosing the middle scale position 
harbors a neutral opinion. A subject's choice of the scale midpoint may result from: 
Ignorance, Uncooperativeness, Reading difficulty, Reluctance to answer, or 
Inapplicability. 

In all the above cases, the investigator's best hope is that the subject will not respond at 
all. Unfortunately, the seemingly innocuous middle position counts, and, when a number 
of subjects choose it for invalid reasons, the average response level is raised or lowered 
erroneously (unless, of course, the mean of the valid responses is exactly at the scale 
midpoint). 

In the absence of a neutral position, responders sometimes tend to resist making a 
choice in one direction or the other. Under this circumstance, the following strategies 
may alleviate the problem: 

1) Encourage omission of a response when a decision cannot be reached. 

2) Word responses so that a firm stand may be avoided, e.g., "tend to disagree." 

3) If possible, help responders with reading or interpretation problems, but take care to 
do so impartially and carefully document the procedure so that it may be inspected for 
possible introduction of bias. 
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4) Include options explaining inability to respond, such as "not applicable," "no basis for 
judgment," "prefer not to answer." 

The preceding discussion notwithstanding, there are some items that virtually require a 
neutral position. Examples are: 

How much time do you spend on this job now? 

1) Less than before 2) About the same 3) More time 

The amount of homework for this course was 

1) too little 2) reasonable 3) too great 

It would be unrealistic to expect a responder to judge a generally comparable or 
satisfactory situation as being on one side or another of the scale midpoint. 

Avoid asking responders to rank responses. Responders cannot be reasonably 
expected to rank more than about six things at a time, and many of them misinterpret 
directions or make mistakes in responding. To help alleviate this latter problem, ranking 
questions may be framed as follows: 

Following are three colors for office walls: 

1) Beige 2) Ivory 3) Light green 

Which color do you like best? 

Which color do you like second best? 

Which color do you like least? 

By carefully evaluating the need of every question used in an instrument and carefully 
wording the responses, you will collect information which will yield more satisfactory and 
meaningful results. 
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